Papers with Transformer decoder

4 papers
Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers (2021.naacl-main)

Copied to clipboard

Hongfei Xu, Josef van Genabith, Qiuhui Liu, Deyi Xiong
Language Models
Challenge: Neural Machine Translation (NMT) has attracted wide attention in recent years.
Approach: They propose a probing-based approach to measure word translation accuracy using transformer layers.
Outcome: The proposed model outperforms previous probing-based translation models.
PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation (2022.acl-long)

Copied to clipboard

Zhe Hu, Hou Pong Chan, Jiachen Liu, Xinyan Xiao, Hua Wu, Lifu Huang
Language Models
Challenge: Existing methods for text generation still suffer from incoherence problems . Neural sequence-to-sequence (seq2sequ) models generate fluent results .
Approach: They propose a novel generation framework that leverages autoregressive self-attention mechanism to conduct content planning and surface realization dynamically.
Outcome: The proposed framework outperforms baseline models and generates more coherent texts with richer contents.
Retrieving Sequential Information for Non-Autoregressive Neural Machine Translation (P19-1)

Copied to clipboard

Chenze Shao, Yang Feng, Jinchao Zhang, Fandong Meng, Xilin Chen, Jie Zhou
Language Models
Challenge: Experimental results show that the Reinforce-NAT system surpasses the baseline NAT system by a significant margin on BLEU without decelerating the decoding speed.
Approach: They propose a sequence-level training method and a Transformer decoder to fuse the target sequential information into the top layer of the decoded Transformer.
Outcome: The proposed model surpasses the baseline NAT system on BLEU without decelerating the decoding speed and achieves comparable translation performance to the autoregressive Transformer model with considerable speedup.
AMR Parsing with Causal Hierarchical Attention and Pointers (2023.emnlp-main)

Copied to clipboard

Chao Lou, Kewei Tu
Semantic Text Processing
Challenge: Abstract Meaning Representation (AMR) is a semantic representation of natural language sentences.
Approach: They propose a new target form of AMR parsing and a model which integrates structural localities into the Transformer decoder.
Outcome: The proposed model outperforms baseline models on four out of five benchmarks in the setting of no additional data.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations